{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ae5bcee9-6588-4d29-bbb9-6fb351ef6630",
   "metadata": {},
   "source": [
    "# L1 Language Models, the Chat Format and Tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c797991-8486-4d79-8c1d-5dc0c1289c2f",
   "metadata": {},
   "source": [
    "## Setup\n",
    "#### Load the API key and relevant Python libaries.\n",
    "In this course, we've provided some code that loads the OpenAI API key for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "19cd4e96",
   "metadata": {
    "height": 132
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import openai\n",
    "import tiktoken\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv()) # read local .env file\n",
    "\n",
    "openai.api_key  = os.environ['OPENAI_API_KEY']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47ba0938-7ca5-46c4-a9d1-b55708d4dc7c",
   "metadata": {},
   "source": [
    "#### helper function\n",
    "This may look familiar if you took the earlier course \"ChatGPT Prompt Engineering for Developers\" Course. \n",
    "\n",
    "Throughout this course, we will use OpenAI's `gpt-3.5-turbo` model and the [chat completions endpoint](https://platform.openai.com/docs/guides/chat).\n",
    "\n",
    "This helper function will make it easier to use prompts and look at the generated outputs. \n",
    "\n",
    "**Note**: In June 2023, OpenAI updated gpt-3.5-turbo. The results you see in the notebook may be slightly different than those in the video. Some of the prompts have also been slightly modified to produce the desired results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1ed96988",
   "metadata": {
    "height": 149
   },
   "outputs": [],
   "source": [
    "def get_completion(prompt, model=\"gpt-3.5-turbo\"):\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=0, # this is the degree of randomness of the model's output \n",
    "    )\n",
    "    return response.choices[0].message[\"content\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73207e48-02a1-4e7e-bf41-ba29ee11ecfc",
   "metadata": {},
   "source": [
    "**Note**: This and all other lab notebooks of this course use OpenAI library version `0.27.0`. \n",
    "\n",
    "In order to use the OpenAI library version `1.0.0`, here is the code that you would use instead for the get_completion function: \n",
    "\n",
    "```python\n",
    "client = openai.OpenAI()\n",
    "\n",
    "def get_completion(prompt, model=\"gpt-3.5-turbo\"):\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    response = client.chat.completions.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=0\n",
    "    )\n",
    "    return response.choices[0].message.content\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe10a390-2461-447d-bf8b-8498db404c44",
   "metadata": {},
   "source": [
    "## Prompt the model and get a completion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e1cc57b2",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "response = get_completion(\"What is the capital of France?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "76774108",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The capital of France is Paris.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b83d4e38-3e3c-4c5a-a949-040a27f29d63",
   "metadata": {},
   "source": [
    "## Tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cc2d9e40",
   "metadata": {
    "height": 64
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pilpolol\n"
     ]
    }
   ],
   "source": [
    "response = get_completion(\"Take the letters in lollipop \\\n",
    "and reverse them\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d2b14d0-749d-4a79-9812-7b00ace9ae6f",
   "metadata": {},
   "source": [
    "\"lollipop\" in reverse should be \"popillol\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "37cab84f",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "response = get_completion(\"\"\"Take the letters in \\\n",
    "l-o-l-l-i-p-o-p and reverse them\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1577c561",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'p-o-p-i-l-l-o-l'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8b88940-d3ab-4c00-b5c0-31531deaacbd",
   "metadata": {},
   "source": [
    "## Helper function (chat format)\n",
    "Here's the helper function we'll use in this course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "8f89efad",
   "metadata": {
    "height": 200
   },
   "outputs": [],
   "source": [
    "def get_completion_from_messages(messages, \n",
    "                                 model=\"gpt-3.5-turbo\", \n",
    "                                 temperature=0, \n",
    "                                 max_tokens=500):\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=temperature, # this is the degree of randomness of the model's output\n",
    "        max_tokens=max_tokens, # the maximum number of tokens the model can ouptut \n",
    "    )\n",
    "    return response.choices[0].message[\"content\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b28c3424",
   "metadata": {
    "height": 183
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Oh, the happy little carrot so bright,\n",
      "In the garden, a joyful sight.\n",
      "With leaves so green and root so firm,\n",
      "It beams with joy, its orange glow affirm.\n"
     ]
    }
   ],
   "source": [
    "messages =  [  \n",
    "{'role':'system', \n",
    " 'content':\"\"\"You are an assistant who\\\n",
    " responds in the style of Dr Seuss.\"\"\"},    \n",
    "{'role':'user', \n",
    " 'content':\"\"\"write me a very short poem\\\n",
    " about a happy carrot\"\"\"},  \n",
    "] \n",
    "response = get_completion_from_messages(messages, temperature=1)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "56c6978d",
   "metadata": {
    "height": 183
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once there was a joyful carrot named Charlie who loved dancing in the sunlight.\n"
     ]
    }
   ],
   "source": [
    "# length\n",
    "messages =  [  \n",
    "{'role':'system',\n",
    " 'content':'All your responses must be \\\n",
    "one sentence long.'},    \n",
    "{'role':'user',\n",
    " 'content':'write me a story about a happy carrot'},  \n",
    "] \n",
    "response = get_completion_from_messages(messages, temperature =1)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "14fd6331",
   "metadata": {
    "height": 217
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In a garden so bright, grew a happy carrot, orange and light.\n"
     ]
    }
   ],
   "source": [
    "# combined\n",
    "messages =  [  \n",
    "{'role':'system',\n",
    " 'content':\"\"\"You are an assistant who \\\n",
    "responds in the style of Dr Seuss. \\\n",
    "All your responses must be one sentence long.\"\"\"},    \n",
    "{'role':'user',\n",
    " 'content':\"\"\"write me a story about a happy carrot\"\"\"},\n",
    "] \n",
    "response = get_completion_from_messages(messages, \n",
    "                                        temperature =1)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "89a70c79",
   "metadata": {
    "height": 387
   },
   "outputs": [],
   "source": [
    "def get_completion_and_token_count(messages, \n",
    "                                   model=\"gpt-3.5-turbo\", \n",
    "                                   temperature=0, \n",
    "                                   max_tokens=500):\n",
    "    \n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=temperature, \n",
    "        max_tokens=max_tokens,\n",
    "    )\n",
    "    \n",
    "    content = response.choices[0].message[\"content\"]\n",
    "    \n",
    "    token_dict = {\n",
    "'prompt_tokens':response['usage']['prompt_tokens'],\n",
    "'completion_tokens':response['usage']['completion_tokens'],\n",
    "'total_tokens':response['usage']['total_tokens'],\n",
    "    }\n",
    "    \n",
    "    original_openai_output = response\n",
    "    return content, token_dict, original_openai_output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "a64cf3c6",
   "metadata": {
    "height": 166
   },
   "outputs": [],
   "source": [
    "messages = [\n",
    "{'role':'system', \n",
    " 'content':\"\"\"You are an assistant who responds\\\n",
    " in the style of Dr Seuss.\"\"\"},    \n",
    "{'role':'user',\n",
    " 'content':\"\"\"write me a very short poem \\ \n",
    " about a happy carrot\"\"\"},  \n",
    "] \n",
    "response, token_dict, original_openai_output = get_completion_and_token_count(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "cfd8fbd4",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Oh, the happy carrot, so bright and so orange,\n",
      "In the garden it grows, a joyful storage.\n",
      "With a leafy green top and a crunchy bite,\n",
      "It brings smiles to all, such a delightful sight!\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "352ad320",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'prompt_tokens': 37, 'completion_tokens': 44, 'total_tokens': 81}\n"
     ]
    }
   ],
   "source": [
    "print(token_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "94f23b06",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<OpenAIObject chat.completion id=chatcmpl-AVJd16QrwRZVbkIl1GLxAPJKSKf16 at 0x7f425c0762c0> JSON: {\n",
       "  \"id\": \"chatcmpl-AVJd16QrwRZVbkIl1GLxAPJKSKf16\",\n",
       "  \"object\": \"chat.completion\",\n",
       "  \"created\": 1732026983,\n",
       "  \"model\": \"gpt-3.5-turbo-0125\",\n",
       "  \"choices\": [\n",
       "    {\n",
       "      \"index\": 0,\n",
       "      \"message\": {\n",
       "        \"role\": \"assistant\",\n",
       "        \"content\": \"Oh, the happy carrot, so bright and so orange,\\nIn the garden it grows, a joyful storage.\\nWith a leafy green top and a crunchy bite,\\nIt brings smiles to all, such a delightful sight!\",\n",
       "        \"refusal\": null\n",
       "      },\n",
       "      \"logprobs\": null,\n",
       "      \"finish_reason\": \"stop\"\n",
       "    }\n",
       "  ],\n",
       "  \"usage\": {\n",
       "    \"prompt_tokens\": 37,\n",
       "    \"completion_tokens\": 44,\n",
       "    \"total_tokens\": 81,\n",
       "    \"prompt_tokens_details\": {\n",
       "      \"cached_tokens\": 0,\n",
       "      \"audio_tokens\": 0\n",
       "    },\n",
       "    \"completion_tokens_details\": {\n",
       "      \"reasoning_tokens\": 0,\n",
       "      \"audio_tokens\": 0,\n",
       "      \"accepted_prediction_tokens\": 0,\n",
       "      \"rejected_prediction_tokens\": 0\n",
       "    }\n",
       "  },\n",
       "  \"system_fingerprint\": null\n",
       "}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_openai_output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65372cdd-d869-4768-947a-0173e7f96335",
   "metadata": {},
   "source": [
    "#### Notes on using the OpenAI API outside of this classroom\n",
    "\n",
    "To install the OpenAI Python library:\n",
    "```\n",
    "!pip install openai\n",
    "```\n",
    "\n",
    "The library needs to be configured with your account's secret key, which is available on the [website](https://platform.openai.com/account/api-keys). \n",
    "\n",
    "You can either set it as the `OPENAI_API_KEY` environment variable before using the library:\n",
    " ```\n",
    " !export OPENAI_API_KEY='sk-...'\n",
    " ```\n",
    "\n",
    "Or, set `openai.api_key` to its value:\n",
    "\n",
    "```\n",
    "import openai\n",
    "openai.api_key = \"sk-...\"\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8f889c1-f2e4-40a5-bd27-164facb54402",
   "metadata": {},
   "source": [
    "#### A note about the backslash\n",
    "- In the course, we are using a backslash `\\` to make the text fit on the screen without inserting newline '\\n' characters.\n",
    "- GPT-3 isn't really affected whether you insert newline characters or not.  But when working with LLMs in general, you may consider whether newline characters in your prompt may affect the model's performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "294dcc54-f941-422e-8602-d8e78a0da093",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
